An Empirical Comparison of Three BoostingAlgorithms on Real Data Sets with Arti cialClass

Boosting algorithms are a means of building a strong ensemble classiier by aggregating a sequence of weak hypotheses. In this paper we consider three of the best-known boosting algorithms: Adaboost 8], Logitboost 10] and Brownboost 7]. These algorithms are adaptive, and work by maintaining a set of example and class weights which focus the attention of a base learner on the examples that are hardest to classify. We conduct an empirical study to compare the performance of these algorithms , measured in terms of overall test error rate, on ve real data sets. The tests consist of a series of cross-validatory samples. At each validation , we set aside one third of the data chosen at random as a test set, and t the boosting algorithm to the remaining two thirds, using binary stumps as a base learner. At each stage we record the nal training and test error rates, and report the average errors within a 95% conndence interval. We then add artiicial class noise to our data sets by randomly reassigning 20% of class labels, and repeat out experiment. We nd that Brownboost proves the least likely to overrt in this circumstance, because the algorithm incorporates an extra parameter which allows it to compensate for noisy examples.